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ABSTRACT 

Will computers that see grid understand what they see revolutionize industry by 
automating the part orientation and part inspection processes? There are two 
obstacles; the expense of computing and our feeble understanding of images. We 
belie v e these obstacles are fast ending. Ta Illustrate what cen be done we 
describe a working program that visually determines the position and orientation of 
silicon chips used in integrated circuits. 


Work reported herein was conducted at the Artificial Intelligence Laboratory, a 
Massachusetts Institute of Technoldgy research program suDported In pari by the 
Advanced Research Projects Agency of the Department of DefeTtse and monitored 
by the Office of Naval Research under Contract number NQ0014-70-A-03 6 2-0005 
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Wm 1 computers. that see and undentand what they see revoMiomie industry by 
automating (he pari orientation and part inspection processes? Predictions have 
been made and hedged, yet very lit tie has been done so far. There are two 
reasons: 

* Computing power $1111 costs too much. 

* Our understanding of real images is loo feeble. 

We believe these obstacles are eroding quickly. There is now every indication 
difficult operations tan be automated soon with dramatic productivity 
m, ~' eases, Far one thing, powerful computers are available now on single 
integrated circuit chips. At the same lime new ideas have emerged that will 
enable machines to fill jobs that are too tedious or dangerous to attract human 
labor. Already one machine in the Held is reading hand-written ZIP codes [1], 
A hot her is visually inspecting printed-circuit cards for etching defects at faster 
than human speed and better than human reliability [2], 

io illustrele what can be done, we describe a working program (hat 
visually determines the position and orientation of tiny silicon chips, a necessary 
prerequisite to establishing electrical contact between the chips and the leads 
p-roh uding from the plastic cases enclosing them. Hyman op era lore perform this 
ta$k lor every transistor and integrated circuit rtisde today by peering through a 
microscope or viewing a TV enlargement. 
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NOW INTEGRATED CIRCUITS ARE MADE 

F-gure I shows a typical integrated circuit <LC.) layout and figure 2 Illustrates the 
way <1 would be connected up Inside a dual inline package (DIP) The 
photographs in figures 3 and 0 show I.C.’s lead-bonded into D.I.P.'s end hybrid 
circuits. Many individual integrated circuits are fabricated together on e single 
two- or three-inch wafer as in figure 5. As it it were e sheet of glass, the water 
is scored with a diamond tool and then split w i„ (o individus, tircui(s rMtfy ((tf 
installat’on in the standard plastic packages. Tiny proba electrodes ere applied Ip 
each chip in turn to see if it works, defective chips are marked with an ink spot. 

Alter a chip is bonded to Ihe subs!rate automatic equipment can altBch 

gold or aluminium wires from pads on the chip to the much larger conductors that 

extend out of the package Unfortunately the target pads ere oof registered 

precisely with the Pines of cleavage and Ihe chips slip a bit as they are laid down, 

thus losing precise alignment Consequenlly the position ami orientation of the 

chip must be determined accurately before Ihe automatic equipment lakes over 

Typically the human operator mov e5 cross-hairs to the tergal tenters using a joy 
slick. 

Newer techniques limit the problem somewhat by using 1C. chips with 
reinforced gold-ball contacts as shown in figure 6. These can be bonded direelfy 
to either lead-frames, as in figure 7 or film-carriers, as in figure 8. The greater 
expense of this process is warranted only because it simplifies the manual 
alignment operation. Since the frame bears a prong lor each pad, only one 
alignment operation Is needed lor each chip, rather than one lor each pad 
connected to Still alignment remains the only operation not yet automated. The 
cost of this is difficult to separate from elher steps, but il ranges from perhaps 3 

cents for simple chips mass-produced abroad to 30 cents for complicated chips 
mad# domsslicaffy. 
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A BORDER SIMPLIFIES PROCESSING 

Quick processing of complicated object* usually requires exploi lotion of special 
properties that make complete image analysis unnecessary. Sometimes slight 
rede vigil: helps. in particular, addition of en aluminium border to integrated circuit 
chips considerably simplifies the orientation task. By happy accident some of tha 
newer Schottky TTL chips now appear with such borders anyway, as seen in 
figures 3 and 10. Since some chip area is sacrificed, generaliilng our method may 
be desirable when handling chips which do not need borders for other reasons. 

THE IMAGE CAN BE PROCESSED QUICKLY BY INEXPENSIVE EQUIPMENT 

Figure 1 L is a diagram of the system used. All modules are standard Items* 
nothing as exotic. Since the overall system was designed for flexible study of a 
broad range qf applied problems, the sophistication and cost far exceed what is 
necessary. We estimate that a system costing 315,000 would do the job easily 
now, wi(h further reduction likely as the cos| of mini- and micro-computer* 
continues down (In fact Ihere is a nice positive feedback loop at work since lead- 
bonding is a substantia! component of the cast of the integrated circuits inside the 
necessary processing equipment.) In addition, part of the equipment could bm 
shared among a number of work-station*. 

Inexpensive nr not, no slow technique would be of much interest to 
industry, It is therefore fortunate that the visual processing can be done in less 
than ,2h second*. To do so requires some work, however. The mismatch Of TV 
vidieen speed and computer speed is the first problem to face. Since a binary 
approximation of Ihe image cannot do the job, several hits of analog to digital 
conversion are needed. Hut multiple gray level conversion at video speed 

requires talented and expensive A to D hardware as well as fancy memory 
buffering. 

,J ickirig off and converting one- point per horiio-ntal scar is the easy 
solution. Using this technique, intensities from one vertical iine of points can be 
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collected for each complete set of horizontal scans. The entire image can be 
acquired by increasing the time between initiation of horizontal scan and sampling, 
thus stepping the vertical tine of sampled points across the image. About 17 
seconds are required to record a complete image of 4BQ. by SI2 pixels this way. 
Fortunately, however, the chip orienting program need not iook at a complete 
image. In fact only (hose poin( 5 indicated in figure 12 are quite adequate and they 
are obtainable from four picture frames in .066 second. The four diagonal lines 
intersect the chip frontiers at eight point. Finding them enables measurements of 
pad position accurate to .5 mils, given that initial errors are within approximately 

ix < 15 mil + Ay < 10 mil, a* < 25 5 


Initial pigment ordinarily « s within lh«o limit*. If the chip is further from 
the expected pcillpn. Nothing j, | 0! , beyond , he (irae rel?uired w ^ p „„ 

of the process following coarse adjustment by the first try. There it a trade-off 
between accuracy end tolerance for targe positioning errors, controlled by the 
placement of the lour diagonal scan lines The positional accuracy ot .5 mite Is 
qu-le satisfactory since the target pads are typically lour mils on a side end the 
connecting wire is one mil in diameter. 


THE IMAGE IS NOT BINARY 

Surprises await the person that thinks LC images are trivial In fact they era 
complicated enough to be interesting but not se complicated as !o require great 
sophistication. Under proper uniform lighting condition* the silicon materia* itself is 
relatively dark and the metalization fighter and not very grainy, but transitions 
between dark and light are not strictly step-like. Figure 13 illustrates tha point. 
The intensity profiles exhibit local irregularities rather than steady change because 
the edge, ot J.C. chips ere ragged and sloped like those of mesas. Specu^r 
reflection point, riddle these edges foiling reliance on simple thresholding. To do 
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the job therefore requires built-in knowledge about edge profile processing, 
Figure 5 Illy states these important steps: 

* Search forward looking lor the chip’s aluminium border. 

The first sign is a rapid rise in intensity exlending over 

several pixels. 

t Next move further ahead looking fQ r the maximum intensity, 

* Search backward to find the minimum. 

* Calculate (max + min)72 and find where the waveform 

crosses this threshhold. This is the hypothesized edge of the 

1C 

The minimum and maximum points tend to be in the midst of plateaus from which 
they differ only slightly in intensity value. Consequently a small amount of noise 
somewhere on a plateau can move an extremum considerably. For this reason* 
averaging between the places where the minimum and maximum are found is not a 
good way to find the edge, Using lire threshhold idea is a standard way to avoid 
the problem. 

Signal hOiae, dust, and scratches cause occasional miscalculation of an 
iril ersection point. This rarely interferes with correct positioning, however* 
because once the boundary is identified as above, built-in verification steps insure 
that the result is sound. These include checks on the difference between 
maximum and minimum, the width of the light and dark plateaus* the width of the 
transition, and so on, as illustrated by the measurements indicated in figure 14. A 
failure initiates further profile analysis and if this too tails, the chip is moved a- 
short distance in the best-guess direction and rescanned. Geometrical tests are 
also performed to ensure that the eight transition points found do indeed lie on a 
rectangle and that the rectangle has the correct dimensions. 
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Notice that tariviAnl mlerwrty patterns may deviate* ccmkfcrabty from 
the ■xpeeted shape. Some, for example, may have an abrupt depression in the 
hPSb .ntenaity plateau caused by a normal £ a P in the border material Others have 
sharp pinnacles where corrida! fractures cause specular reflections. Template 
matching schemes are hard pressed by such problems because if is herd to supply 
them with specific knowledge about particular anomalies. Thankfully, building 
knowledge into procedures is immensely easier. 

Aitoeetter the approach resembles Shi™’* treatment of scene anoints 

' S * W0, ' d »' blocl1 '' »«*('». '<■"£ * love rite ot workers tn artificial 
intelligence [3] indeed Shirei’s feature point program is a generally important 

source of ideas with which everyone in image processing should be lamitiar. 
CALIBRATION IS EASY 


The program needs the relationship between pixels and the step sice of the 

positioning table. There are several factors which ioltuence this; pixel sine varies 

With each vidicon tube; the pixel size differs in x and y, there is seme relative 

rotation between the camera and the table; and the x and y axes in the camera 

are not exactly orthogonal. The Mowing linear transformation ecoomts for these 
fmlon; 


] X V 


a b 




§r 




y. 


c d 


Yi i 


Where x, and y, are pixel units and x, end y, are table step units. 

Matrix elements a, b, c. and d are easily oblarned by observing the displacement in 
the vidicon which results from stepping a standard 1C. through known calibration 
points. Tha inverse matrix then gives correct table movement for observed 
offsets. Refinement of this calibration process aimed ef compensating for non- 
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linear distortion in the vicScon canere would increase accuracy, although further 
improve merit dae? not seem ncees&ary flt the moment. 

AN INDUSTRIAL REVOLUTION IS NEEDED 

We believe this application and others like it are the vanguard of b new kind of 
productivity technology, one oriented toward facing diflieult industrial vision and 
manipulation tasks squarely. Past soluticns always invoked using people or costly, 
inflexible special purpose automation But people are moving increasingly toward 
the service sector and away from the boredom o! the assembly line- Many q\ the 
things they do are difficult to automate because spatial purpose automation costa 
too much or cannot work. Thus the need for progress in vision-based productivity 
technology is acute. 

Even Ip the electronics industry, well known for Its aggressive pasture 
with respect to new ideas, there are many other examples of expensive, manual 
assembly, inspection, and repair operations. Here are just a tews 

Inspection of Printed Circuit Cards; 

When a large circuit hoard comes out of the etching bath it is 
likely to have many crack and bridging defects. These must 
be found and corrected before component insertion. Later on 
the board must bo inspected again after wave soldering, 

V again with many hand corrections necessary. The Japanese 
have made some progress cm this problem \2\ 

Assembly of Electronic Circuits; 

Not alt components can bo handled by automatic insertion 
machines. Visual inspection is required in any cats. Largs or 
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odd-shaped components such as transformers and components 
with (loppy leads present Special problems. 


Repair ot Assembled Modules^ 

W,th larger hoard areas the yield ot wording circuits is low, 
The new” computer is actually a repaired computer 
debugged by an expensive,, trained technician. 


Eventually machine vision is hkeiy to increase productivity in such diverse 
activities as undersea and underground mining farming chores, surveillance of 
dengereus machines, package sorting, custodial care, planetary exploration, end the 
manufacture of shoes, clothes, and other consumer goods. Everyone knows that 
these are lough problem* to hand over to flexible computer-based systems. Until 
rvow the technology has not been adequate and computation cost? have not been 
low enough to move maoh-nes with these capabilities out of the laboratory and into 
the manufacturing process. The great strides that have been made In automating 
continuous processes like oil refining and paper making, or in discrete parts 
manufacturing for that matter, have yet to be matched in situations requiring 
mechanical assembly or human quality visual inspection. It is time to cetch up. 
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Figure I 



Sketch ot Integrated circuit silicon chip showf™ dark silicon 
e-dgldg, bright aluminium border and connecting pads, 
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Figure 2: Sketch of tvpFcal si^teen~piri dual -Fnl Ipe-packege il lustrating 
how the Entegrit*<l clfCult Chip Is conoctt^d to the leads 
emhedded E n package, 




























































figure 3- Typica! integrated circuit chips iead'bOndEd Pnta fourteen-p-1 r> 
dua I-Er I fne-packageS using one-roEl aluminium wire. 



Figure ^ r Typical integrated circuit Chips Tead-bonded Into hybrid circuits 
usfng opic-mit gold wire. 
















Figure 5; Standard 3 “ wafer, only defective 
chips have been left behind. 



FEgure 7: Lead-frame etrip of ten-rfi!! 
gold-plated bra-i'i- 


FF^urc: 0: Fi In-carrier strip. 






































































































Figure 9; 


Hicrograph and phot* of TV non! tor 
chip with bright a P urn i n f-unt border. 


image of integrated circuit 
Not* dark gap in the border. 



Figure 10; 


Micrograph and photo of TV monitor 
Chtp with bright afurn inturn border, 
C&USed by probing electrodes. 


image of integrated circuit 
Note high- | ighfs on pads 




















F 3 gurc lli B kook-d lag ram of Syste-n used For laboratory demons t ration of 

automatic positioning and orienting of integrated circuit cKipS 
using visual input. 



















































figure 12; The four diagonal scan (fries superimposed on a sketch of the 
image of an integrated circuit chip, Circle? correspond to 
the eight edge points feu^d by the program. 
















Figure IJ: 11 ] us t rat [on Of the waveform an<ct measurements taken to determine 
a hypothesized cd'ge position* 



Figure 14i I t lustration of the waveform and some of the measurements taken 
to verify a hypothesized edge positIon* 















































